Papers with multi-task settings
Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP (2021.eacl-demos)
Copied to clipboard
| Challenge: | Multi-task learning (MTL) has become a standard repertoire in natural language processing (NLP) it enables neural networks to learn tasks in parallel while leveraging the benefits of sharing parameters. |
| Approach: | They propose a toolkit for fine-tuning contextualized embeddings in multi-task settings. |
| Outcome: | The proposed toolkit supports a variety of natural language processing tasks . it enables neural networks to learn tasks in parallel while leveraging the benefits of sharing parameters. |
Deploying Multi-task Online Server with Large Language Model (2025.coling-industry)
Copied to clipboard
| Challenge: | In the industry, numerous natural language processing tasks are deployed online . traditional approaches tackle each task separately by its own network and pipeline . |
| Approach: | They propose a three-stage multi-task learning framework for large language models . it involves task filtering, fine-tuning on high-resource tasks, and finally fine- tuning on all tasks . |
| Outcome: | The proposed framework reduces up to 90% of overhead while reducing latency and resource usage. |
Modeling Task Interactions in Document-Level Joint Entity and Relation Extraction (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing work on document-level relation extraction has focused on end-to-end setting that extracts global entities and relations jointly. |
| Approach: | They propose to introduce a two-way interaction between COREF and RE that is specifically designed to leverage task characteristics, bridging decisions of two tasks for direct task interference. |
| Outcome: | The proposed model achieves the best performance by up to 2.3/5.1 F1 over the baseline. |
CoMoE: Contrastive Representation for Mixture-of-Experts in Parameter-Efficient Fine-tuning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, mixture-of-experts (MoE) is underutilized on heterogeneous datasets, ignoring the fact that experts may learn similar knowledge. |
| Approach: | They propose a method to promote modularization and specialization in MoE by specializing functionalities into different experts and sparsely activating them appropriately. |
| Outcome: | The proposed method improves the capacity and specialization of mixture-of-experts (MoE) by sampling from activated and inactivated experts in top-k routing. |
MoDE: Effective Multi-task Parameter Efficient Fine-Tuning with a Mixture of Dyadic Experts (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent efforts have explored mixtures of LoRA modules for multi-task settings, but this study reveals redundancy in the down-projection matrix of these architectures. |
| Approach: | They propose a method to share down-projection matrix across tasks and employ atomic rank-one adapters coupled with routers that allow more sophisticated task-level specialization. |
| Outcome: | The proposed method outperforms state-of-the-art models on a SNI benchmark and provides a practical solution for deploying lightweight models. |
CAN: Constrained Attention Networks for Multi-Aspect Sentiment Analysis (D19-1)
Copied to clipboard
| Challenge: | Existing methods for aspect-specific sentiment classification are noisy and downgraded performance. |
| Approach: | They propose a constrained attention network to regularize attention for multi-aspect sentiment analysis by orthogonal regularization on multiple aspects and sparse regularization for each single aspect. |
| Outcome: | The proposed approach outperforms state-of-the-art methods on two public datasets and extends to multi-task settings. |
Impartial Multi-task Representation Learning via Variance-invariant Probabilistic Decoding (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods focus on balancing loss or gradients but fail to address this issue due to the representation discrepancy in latent space. |
| Approach: | They propose a framework that harmonizes representation spaces across tasks to ensure impartial learning by harmonizing representation spaces. |
| Outcome: | The proposed framework outperforms 12 representative methods under the same multi-task settings, especially in heterogeneous task combinations and data-constrained scenarios. |
From Heads to Neurons: Causal Attribution and Steering in Multi-Task Vision–Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing models focus on single tasks, limiting comparability of neuron importance . ranking strategies overlook how task-dependent information pathways shape write-in effects of feed-forward network (FFN) neurons. |
| Approach: | They propose a gradient-free framework for task-aware neuron attribution and steering in multi-task vision-language models. |
| Outcome: | The proposed framework outperforms existing methods in identifying task-critical neurons and improves model performance after steering. |